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LOW- VOLTAGE, VERY-LOW-POWER 
CONDUCTANCE MODE NEURON 

CROSS-REFERENCE TO RELATED 
APPLICATION 

This application is a continuation of U.S. patent applica- 
tion Sen No. 08/731,426 filed Oct. 15, 1996 and allowed 
Sept. 13, 1999 now U.S. Pat. No. 6,032,140. 

TECHNICAL FIELD 

The present invention relates to a low-voltage, very-low- 
power conductance mode neuron. 

BACKGROUND OF THE INVENTION 

As is known, artificial intelligence systems feature neural 
networks for performing complex tasks, in particular, for 
texture analysis, morphological kernel filtering in facsimile 
transmission, vehicle tracking systems, pattern recognition, 
hardware simulation in neural CAD systems, and prepro- 
cessing in optical character recognition applications. 

Neural networks employ components known as neurons, 
similar to the biological elements of the same name and 
based on the addition and subtraction of appropriately 
weighted inputs, and for which various mathematical for- 
malisms have been devised. Reference is made in the present 
invention to binary neurons according to the McCulloch- 
Pitts model, which stands out for the precision and elegance 
of its mathematical definition, and according to which, the 
output may assume only two binary values **0" and "V and 
operates under a discrete time assumption, with predeter- 
mined neuron thresholds and weights. Each neuron com- 
prises a processing element with a number of synaptic input 
connections and one output; and input and output signal flow 
is considered one-way. 

In the classification phase neural network parameters are 
fixed and the neural network executes the recognition or the 
analysis starting from the information contained in the 
topology and in the weights of the neural network. FIG. 1 
shows a symbolic representation of the McCulloch-Pttts 
model, in which x 2 , Xj, . . . , x, are the inputs, w 2 , w^ . . . 
, w- the weights, and O the output. The neuron is represented 
by a node defining function f, which, when applied to the 
weighted inputs, supplies the output according to the equa- 
tion: 

Typically, the f function compares the sum of the 
weighted products of the inputs with a threshold, and, 
depending on the outcome of the comparison, determines 
the binary value of the output 

Various solutions are known for implementing neural 
networks, as described for example in S. Satyanarayana et 
al., "A Reconfigurable VLSI Neural Network," IEEE Jour- 
nal of Solid-State Circuits 27:1, January 1992; B. E. Boser 
et al., "An Analog Neural Network Processor with Program- 
mable Topology," IEEE Journal of Solid-State Circuits 
26:12, December 1991; and A. Kramer et al, "EEPROM 
Device As a Reconfigurable Analog Element for Neural 
Networks," IEDM, 1989, pp. 259^262. 

All known solutions, however, involve a trade-off 
between power consumption and precision, require a large 
integration area, and are complex in design. Moreover, all 
known prior art solutions make it necessary to choose 
between solutions designed for high speed but requiring 
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high power (current mode computation) and solutions 
designed for low power but operating at low speed (charge 
computation mode). 

Analog implementations of Neural Network Architectures 
5 provide a framework for computation which is more effi- 
cient than standard digital techniques for certain problems. 
Typically, implementations of analog neural networks have 
been based on the use of either current or charge as the 
variable of computation. 

10 

SUMMARY OF THE INVENTION 

The present invention provides a new class of analog 
neural network circuits based on the concept of 
conductance-mode computation. In this class of circuits, 
accumulated weighted inputs are represented as 
conductances, and a conductance-mode neuron is used to 
apply nonlinearity and produce an output. In the hardware 
implementation of neural networks it is important to con- 
sider flexibility and power consumption in order to satisfy a 
wide range of applications. The present invention has been 
to focus on circuits which consume very little power per 
connection allowing for a high number of connections per 
neuron. 

25 It is an object of the present invention to provide a 
compact, low-power neural network with a high input range 
that operates at high speed. It is adaptable to different (even 
low-voltage) supply conditions and it requires no additional 
interfacing to be compatible with digital environments. It 
3Q may also be used in portable devices. 

One advantage of the present invention is that it imple- 
ments synapses by a simple circuit based on a pair of 
floating-gate transistors, providing both analog multiplica- 
tion and weight storage with low power consumption, high 
35 density and high precision. Both the weight storage and 
analog multiplication are implemented concurrently in a pair 
of floating gate transistors. 

The advantages of this class of circuits are twofold: firstly, 
conductance-mode computation is fast - circuits based on 
40 these principles can compute at about 5-10 MHz; secondly, 
because conductance-mode computation requires the mini- 
mum charge necessary to compare two conductances, its 
energy consumption is self-scaling depending on the diffi- 
culty of the decision to be made. The computing precision of 
45 these circuits is high. Test results on a small test structure 
indicate an intrinsic precision of 8-9 bits. 

A preferred, non-limiting embodiment of the present 
invention will be described by way of example with refer- 
ence to the accompanying drawings. 

50 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 shows a symbolic representation of a neuron 
according to the McCuUoch-Pitts model. 
55 FIG. 2 is a schematic showing use of a flash EEPROM 
device as it will function as a programmable, switched 
conductance. 

FIG. 3 is a schematic of a differential, summed synaptic 
conductance neuron. 
60 FIG. 4A is a block diagram of the principle blocks of the 
present invention. 

FIG. 4B is a block diagram of the present invention. 
FIG. 4C shows an overall circuit diagram of a neural 
65 network circuit according to the present invention. 

FIGS. 5 to 7 show plots of a number of quantities of the 
circuit in FIG. 4C. 
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FIG. 8 shows a schematic of a test structure. course, future improvements in programming and storage 

FIG. 9 shows a block diagram of a new test structure. technology may permit long term storage of even more 

r-i^> ia ~f precision. This corresponds to 8 bits to 5 bits (256 to 32 

FIG. 10 shows an intrinsic precision curve ot test struc- f „. . * ^ , . >. 

levels) over a 2V mput dynamic range. While current 

5 technology will permit flash EEPROMs to be used for long 

FIG. 11 shows a precision curve with 1 k devices of term sto rage, up to one year for 5 mV to 8 mV threshold 

common mode. variations, and for 64 mV variations up to 10 years, other 

FIG. 12 shows a precision curve with OS k devices of conductance storage devices may be used Usage of two 

common mode. devices adds an additional sign bit to increase the total to 9 

FIG. 13 shows bit precision curve versus LSB voltage for 1Q to 6 bits per weight. 

1 k devices of common mode. The present invention uses a conductance comparator as 

FIG. 14 is a diagram of neuron functionality at 10 MHz. a neuron t0 Wty * e "activation function" of the neuron that 

FIG. 15 is a power consumption measurement driven by «" verts me tota | conductance input to a binary 

a 1.7 MHz clock with a 5 volt power supply. Integrated ™ lt *& output suitable for long range communication and 

supply current through a 9 k Ohm resistor is shown. 15 ^P tal Dircctlv . «""P»™* two conductances 

allows the present invention to compute using very little 

DETAIL DESCRIPTION OF THE INVENTION energy, reducing the overall power consumption. 

The present invention uses a single pair of Flash- FIG. 4A illustrates a circuit that performs the neuron 

EEPROM devices for both analog storage and analog com- computation. The circuit is a conductance measuring device 

putation. Basically, the computational concept is to make 2Q that comprising two principal blocks: conductance synapses 

use of a floating-gate device as a programmable switched 91 and a neuron 5. FIG. 4B shows a current buffer 4 

conductance. By storing one analog value as the threshold of decoupling the synapses 91 from the neuron 5 where the 

a floating gate device and applying a second digital value on neuron performs the comparison of conductance of the 

a gate terminal of the floating-gate device, the conductance synapse 91. The circuit may also include a latch 6 to digitize 

of the device can be either zero (oft) or a pre-programmed ^ an output of the circuit. In one embodiment of FIG. 4A and 

analog value. In this way, a single programmable device is 4B, each conductance synapse is composed of a single 

able to implement a multiplication of a single binary value conduction element whereas in n alternative embodiment, 

(Vg of programmable transistor) by an analog value (Vh^- each conduction synapse is composed of two conduction 

VjtJ- The single programmable device is a conductance elements. 

synapse whose weight is computed by the neuron when the ^ FIG. 4C indicates a neural network 1 comprising a first 

input is taken high. exciting (positive-weighted) synapse block 2; a second 

In one implementation, a differential input scheme com- inhibiting (negative-weighted) synapse block 3; a decou- 

p rising two conductance summing lines allows weights to be pling stage (buffer) 4; a comparing stage (neuron) 5; and a 

stored in the form of a pair of conductances. The weights are latch circuit 6. 

either "positive" or "negative." For each two transistor 35 Synapse block 2 has a number (n) of inputs 11 supplied 
synapse, one device is programmed to be always off, inde- with (n) input signals x 19 x^ . . . , a set of (n) positive 
pendent of input, and the other device is programmed to be value synapses 15; and an output 12 which is the sum of the 
on only when the input is high. Synapse polarity (positive or current through all positive synapses through which flows 
negative) is determined by which summing line to which the current l x . Synapse block 3 comprises an equal number of 
on device is connected. A conductance comparison neuron ^ inputs 13 supplied with the same n input signals (each input 
can then compare the sum of the total "positive** conduc- signal is supplied in parallel to an input 11 of exciting 
tance to the sum of total "negative" conductance and make synapse block 2, and to an input 13 of inhibiting synapse 
a decision on the polarity of the total weighted inputs block 3); a set of (n) negative values synapses 17; and an 
(conductance). A binary output is provided whose value is output 14 through which flows current 1 2 . 
base on the results of the comparison 45 in this embodiment, the synapses 91 are implemented 
FIG. 3 shows a schematic of a electrical representation of with a positive-weighted synapse block 2 and a negative- 
such a conductance neuron device's functionality. Each weighted synapse block 3; however, other conduction 
collection of the total "positive" conductance devices 2 or schemes within the context of the present invention use only 
the total "negative** conductance devices 3 constitutes the one type of weighted synapses. While certain advantages are 
synapses of a neuron. The conductance or weights of these 50 obtained using positive and negative synapses in a compari- 
synapses are determined by a threshold voltage ( Vt) pro- son mode, this is not required in all implementations making 
grammed on the neuron. The precision to which this thresh- use of the present invention. 

old voltage can be controlled gives the effective bit- Synapse block 2 comprises a number of synaptic elements 

equivalent precision of the synapse weight. 15 comprising memory cells 15 — in this case flash- 

The analog programmed value is controlled by the pro- 55 EEPROM cells — of the same number as inputs 11. More 

grammed threshold voltage. In this embodiment, the actual specifically, each synapse cell 15 has a grounded source 

value is equal to the digital on voltage, minus the Vy% V r terminal, a gate terminal connected to a respective input 11, 

is programmed to a value as selected by the user, a control and a drain terminal connected to the drain terminals of the 

circuit, or a program. These values can be provided by any other cells 15 and to output 12. Similarly, synapse block 3 

number of accepted techniques, including via a hardware 60 comprises a number of synaptic elements comprising 

data transfer, software program, or internal adaptation. The memory cells 17 (of the same number as cells -15); and each 

internal adaptation may occur by use of neural algorithms cell 17 has a grounded source terminal, a gate terminal 

implemented or feedback loops on the same chip so that self connected to a respective input 13, and a drain terminal 

programming and modification can be controlled on a single connected to the drain tenninals of the other cells 17 and to 

chip without need for a host processor. 65 output 14. As can be seen by viewing FIG. 4C, the sources 

It is possible to program the threshold of a floating-gate of each conductance element 15 and 17 are tied to a common 

device to a precision in the range of 8 mV to 64 mV. Of voltage potential, which in one embodiment is ground. 
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Flash-EEPROM memory cells are used for cells 15 and 
17 is one embodiment of the present invention. The use of 
floating gate technology for efficient long term analog 
storage is well known in the art, especially in neural network 
implementations. See, for example, an article by A. Kramer 5 
et ah, entitled "Flash-based Programmable Nonlinear 
Capacitor for Switch Capacitor Implementations for Neural 
Network," IEDM Technology Digest, pp. 17.6.1-17.6.4, 
December 1993. However, other possible conductance stor- 
age devices, such as EEPROM memory cells, dynamic 10 
floating gate devices, non-programmable, non-floating gate 
devices (i.e., standard MOS transistors of selected size) or 
other types of storage elements suitable for generating 
conductance, are used in other embodiments. 

Referring to FIG. 4C, decoupling stage or buffer 4 com- 15 
prises a first diode-connected N-channel MOS transistor 20, 
and a second and third N-channel MOS transistor 21, 22. 
More specifically, first transistor 20 has the drain terminal 
connected to a supply line 25 at supply voltage \ DD via a 
current source 26 for generating current l b ; the gate terminal 20 
(node A) connected to its own drain terminal and to the gate 
terminal of transistors 21, 22; and the source terminal 
grounded. Second transistor 21 has the source terminal 
connected to output 12 of synapse block 2, and a drain 
terminal defining a first output 27 of buffer 4. Third tran- 25 
sistor 22 has the source terminal connected to output 14 of 
synapse block 3, and a drain terminal defining a second 
output 28 of buffer 4. 

A reset transistor (represented in FIG. 4C by a switch 29) 
is located between the gate and source terminals of transistor 30 
20, and presents a control input 29a. In the presence of a 
control signal at input 29a (voltage V^), switch 29 grounds 
the gate terminals of transistors 20-22, and hence the drain 
terminals of memory cells 15, 17, to eliminate the current 
through the memory cells when network 1 is not used. 35 

Neuron 5 comprising a conductance comparator which 
senses the difference between the synapse blocks 2 and 3 
separated in positive and negative weights. This conduc- 
tance mode neuron is a suitable building block for large- ^ 
scale array-based analog neural network implementation 
and has been designed using a mixture of analog and digital 
subcircuits (mixed-mode). The neuron circuit uses analog 
weighting and analog computation internally to reduce sili- 
con area and power consumption while its data inputs and 45 
outputs are digital. One of the concepts underlying the 
present invention is to make use of conductance summing 
for performing computation and for converting an analog- 
valued conductance signal into a binary-valued output signal 
suitable for communication. The use of digital I/O signals 5Q 
greatly simplifies integration at the system level In one 
embodiment, the I/O is digital; but, alternatively analog I/O 
may be used. 

Neuron 5 comprises a symmetrical first and second sec- 
tion 31, 32 connected respectively to outputs 27, 28 of buffer 55 
4 and cross-connected to each other. Section 31 comprises 
three P-channel MOS transistors 35-35 with the source 
terminals all connected to supply line 25, and the gate 
terminals connected to one another; transistor 33 is diode- 
connected, forms a current mirror with transistors 34, 35, 50 
and has the drain terminal connected to a node 36 in turn 
connected to output 27 of buffer 4; transistor 34 has the drain 
terminal connected to a node 37 of section 32; and the drain 
terminal of transistor 35 forms output 38 of the neuron, and 
is connected to latch circuit 6. 65 

Similarly, section 32 comprises three P-channel MOS 
transistors 43-45 with the source terminals all connected to 
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supply line 25, and the gate terminals connected to one 
another, transistor 43 is diode-connected, forms a current 
mirror with transistors 44, 45, and has the drain terminal 
connected to node 37 in turn connected to output 28 of buffer 
4; transistor 44 has the drain terminal connected to node 36 
of section 31; and the drain terminal of transistor 45 forms 
output 39 of the neuron, and is connected to latch circuit 6. 

Reset transistors (represented in FIG. 4C by switches 40, 
41) are located between supply line 25 and the gate terminals 
of transistors 33-35 and 43-45 respectively, and present 
control inputs 40a, 41a. In the presence of control signals at 
inputs 40a, 41a (voltages VR), switches 40 and 41 short the 
gate terminals of transistors 33-35 and 44-45 to the supply 
and so turn off the transistors when network 1 is not used. 

Latch circuit 6 is a conventional type comprising three 
N-channel MOS transistors 47-49. More specifically, tran- 
sistor 47 has the source and drain terminals connected to 
outputs 38, 39 of neuron 5, and the gate terminal forms a 
control input 50 supplied with an enabling signal EN; 
transistor 48 has the drain terminal connected to output 38 
of neuron 5, the gate terminal connected to output 39 of 
neuron 5, and the source terminal grounded; and transistor 
49 has the drain terminal connected to output 39 of neuron 
5, the gate terminal connected to output 38 of neuron 5, and 
the source terminal grounded. 

A single synapse can be viewed as being comprised of one 
synapse element 15 from block 2 and one synapse element 
17 from 7. These two synapse elements have a common 
input and form a single synapse together. As previously 
stated, in a preferred embodiment, one of these elements will 
be programmed on to the desired conductance value when 
the input is high and the other will be programmed to always 
be off, regardless of the input value. The conductance of 
respective elements 15 will be summed to provide a positive 
conductance signal and the conductance of respective ele- 
ments 17 will be summed to provide a negative conductance 
signal These are compared to provide a binary output, the 
comparison being one technique to provide a fast compu- 
tation. 

Of course, the conductance synapses of this invention 
could also be composed of a single conductive element for 
each synapse, whose value is computed upon an input signal 
being provided. This could be implemented, for example, by 
using only a single block 2 and compare the output to some 
reference value for a digital output; or alternatively, to 
provide the neuron output as an analog output Alternatively, 
a synapse comprised of two devices as generally described 
can be programmed so that both devices are on when the 
input is high. In this case, the conductance value of the 
synapse corresponds to the difference in conductance value 
between the two elements. This embodiment can be useful 
in providing a circuit which is more insensitive to the effects 
of temperature variations on effective threshold voltage. In 
a further alternative, rather than comparing the conductance 
value with respect to ground, elements 15 and 17 could have 
their respective sources tied to a respective selected voltage 
potential other than ground. In one embodiment, all the 
elements 15 of block 2 are tied to a first selected voltage 
potential and all elements of block 3 to a different selected 
voltage potential. This could provide increased differences 
in the output signals for improved conductance measure- 
ment comparison. Thus, while, the conductances are shown 
measured with respect to ground in FIG. 4C, they are 
measured with respect to any selected fixed voltage, includ- 
ing different voltages for different individual elements 15 
and 17, or entire groups of elements within blocks 2 and 3. 

In one embodiment, each synapse is a single conductance 
element implemented by floating-gate or other storage 
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devices. In an alternative embodiment, each synapse is a pair voltage at outputs 12, 14 of synapse blocks 2, 3, so that 

of conduction elements implemented by a differential pair of memory cells 15, 17 operate in the triode region, and 

floating gate devices. This presents several design con- network I can also operate at low supply voltages and low 

straints to the current buffer Firstly, to minimize disturb power. 

programming it is helpful to keep the drain voltage of the 5 Operation of buffer 4 will now be described mathemati- 
floating-gate device of the synapse as low as possible. In one cally with reference to exciting summing line 12 from 
embodiment, this is less that 100 mV, but higher voltages positive synapse block 2; the same also applies to inhibiting 
could be used in other technologies or designs. Secondly, as summing line 14 from inhibiting synapse block 3. 
the present invention intended to use many synapses and ^ ^ vq1 d betweeQ me mm ^ te terminals 
each has parasitic dram capacitance on the order of 2 fF, the 10 of a MQS transistor cq ^ s me threshold voltage (ininimum 
drain terminal of the synapse will respond slowly if it must voUa ^ at ^ transistor ^ turned on) plus the over- 
undergo large voltage swings. To have a neuron with a large ^ ^ (vq1 m excess of the mreshold value)j the 
dynamic range output voltage and high speed, it is desirable following equation applies for transistor 20: 
to decouple the drain node of the synapses from the neuron. 

The number of synapses per neuron can be any number in 15 K^V^^Vov^+v^o CO 

one embodiment there are from 1 to 2000 memory cells 15 

and 17 in each of synapse blocks 2 and 3. where V A is the voltage drop between node A and ground; 

One advantage of the present invention is that it can and Ywft V' 20 ^V 20 f e resp ? c ^ vel y * e f te " 

correctly and rapidly compare the synaptic conductances, voltage drop, overdrive voltage and threshold voltage 

. „ r * n 't Q _ jL^u ^„*^™T 20 of transistor 20. 
even in the presence ot a large common mode conductance. , . _ 

Common mode conductance corresponds to that part of the M °™Yf ' , " "2* V ° 1U f ^ ° UtP ^ 

signal which is conductive for both the negative and positive ground (the vohage between the drain and source terminals 

conductance summing lines. The difference conductance 06 *" 

signal corresponds to the residual conductance present on v^-v^v^ -v OK ^4-v^ ^ (2) 
either the positive or negative summing line, whichever is 25 05,21 OY ** 21 1 
greater. The common mode conductance can vary over a where V OV21 and V tAai are respectively the gate- 
wide range, from as few as no synapses on as common to as source voltage drop, overdrive voltage and threshold voltage 
high as nearly all synapses on (i.e., 1000 or 2000). Compute Q f transistor 21. 

precision corresponds to the residual conductance signal Since transistors fabricated using the same technology 

needed to correctly control neuron output in the presence of present roughly the same threshold voltage, so that 

maximum common mode conductance. -V dk;21 , substituting (1) in (2) with a few straightforward 

Memory cells 15, 17 of synapse blocks 2, 3 of network 1 calculations gives: 
are programmed according to the weight w f they are to 

represent, and, more specifically, may be programmed at 35 v^»v 0 ^-v 0K21 (3) 

different threshold levels to present a given conductance for ^ . _ t , 
each input voltage value. In the simplest case, they may be Transistor 21 may be appropriately sized to present a 

programmed to define only two binary levels (cell on or off neghgible overdrive voltage in relation to that of transistor 

for a given input voltage value at inputs 11, 13 respectively), 20 > 50 thal ( 3 > bGCOmes: 

and the input voltage (input signals x l9 . . . , xj may assume ^ v v (4) 

only a predetermined number of discrete values (e.g., two, ° VJO 

binary high or low; or four). Alternatively, the cells may be and the voltage between the drain and source terminals of 

programmed to store a number of different threshold values, ceDs 15 equals the overdrive voltage of transistor 20. 
and the input voltage may vary continuously within a For transistor 20, which operates m the normal region, the 

predetermined range to actually evaluate the conductivity of 45 following equation applies: 
the cells in the presence of data input signals. 

Consequently, when input signals x l9 . . . , X„ are supplied h-^Ycsjo- ^^^(Vov^ (5) 

to inputs 11, 13, the current through each of memory cells 

15, 17 is related to the conductance of the cell, which is the where h * current transistor 20 and equal to the 

prograrnmed, stored weight. Tne current I,, I^ through each 50 t*™* generated by source 26; and K is a proportion 

node 12, 14 equals the sum of the currents in all the cells of constant. 

the respective synapse blocks 2, 3, and as such is related to Substituting (4) in (5) with a few straightforward calcu- 

the total conductance of the synapse. lations gives: 

Buffer 4 provides for decoupling, and more specifically v^~^TJK (6) 

for ensuring the neuron only sees the capacitance of tran- 55 

sistors 21, 22 (the capacitance between the drain and gate Consequently, by appropriately sizing current source 26, 

terminals phis the capacitance between the drain and source the drain-source voltage drop of cells 15 (and 17) may be 

terminals of transistors 21, 22) and not the total capacitance predetermined, and more specifically, may be made less than 

of cells 15, 17, which, in view of the large number of cells the gate-source voltage drop of cells 15. 
that may be provided (even as many as a few thousand 50 Neuron 5 is a conductance sensing circuit that performs a 

synaptic elements per neuron), may be extremely high. This comparison between two different conductances corning 

provides for much faster evaluation by network 1, as from the synapse devices. Neuron 5 provides for comparing 

explained in more detail herein. the respective currents l l9 and I 2 of the exciting and inhib- 

To reduce the power consumption in the computation it is iting synapses, and, together with latch 6, for generating at 

better that all the devices implementing synapses work in the 65 output O (node 39) a digital signal, the value of which 

triode region. However, this is not always required. In the indicates which of the two currents is the higher. More 

present invention, buffer 4 provides for maintaining a low specifically, it operates as a feedback current sensor, wherein 
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a transient state is observed when the circuit is turned on, equals I C +AI, current I 34 equals I c , and currents I 43 , 144 are 

and at the end of which practically all the current flows into zero. The FIG. 6 graph shows output voltages O and at node 

section 31 or 32 connected to the synapse with the higher 38 (voltage V 38 ). 

output current, and the other section is practically turned off The off state of the section connected to the lower-current 

altogether. A digital output signal level is thus reached very 5 synapse (section 32) disconnects the respective output node 

quickly. (39) from the supply; whereas the high-conductance condi- 

For an easy decision, namely, a large difference between tion of the section connected to the higher-current synapse 

the conductance values of blocks 2 and 3, the final digital (section 31) latches the respective output node (38) to supply 

output is reached very quickly with very low current con- voltage V^. Consequently, in the example shown, upon 

sumption. Thus, even though many synapse elements may 10 signal EN switching to low and turning off shorting tran- 

be turned on, the actual decision itself is based on a sistor 47, transistor 49, whose gate terminal is connected to 

difference and thus is very quick and low in power if that the turned -on section 31, is turned on and grounds output O; 

difference is great. If the decision is difficult, namely, the and the other transistor 48 remains off. Conversely, if 

value of currents 1 ± and are nearly equal, slightly more inhibiting synapse block 3 presents a lower conductance 

time may be required to make the decision with slightly 15 (higher current) than exciting synapse block 2, section 32 

more current usage. Accordingly, the present invention and transistor 48 are turned on, and section 31 and transistor 

provides the advantage that easy decisions are made quickly 49 are turned off, so that output O is high and latched to 

with low power consumption and the more difficult deci- supply voltage W OD via transistor 45. 

sions are permitted the extra time needed to ensure a correct The advantages of the neural network according to the 

output. 20 present invention are as follows. By implementing the 

To explain the working of neuron 5, currents \ x and \ will synapses using memory cells whose conductance varies 

be assumed to present a common term I c , and to differ by a according to the programming level, the synapses may be so 

differential term AI. For example, exciting synapse block 2 biased as to present a low output voltage (nodes 12, 14), thus 

will be assumed to generate a current I i =I c +AI, and inhib- permitting network 1 to be used in low-voltage conditions 

iting synapse block 3 to generate a current I 2 =I C - 25 and hence also in portable devices and systems. Moreover, 

When reset signal V R switches to eliminate the shorted low-current operation of the synapses (100 pJ in the case of 

condition of buffer 4 and neuron 5, cells 15, 17 begin a thousand inputs) provides for low-power operation of the 

drawing current, and currents l l9 and ^ begin flowing network as a whole, which may therefore also be used in 

through sections 31, 32 to turn them on. At this transient applications in which voltage and power resources are 

stage, however, the current mirrors of the two sections are 30 limited. 

not turned on at the same speed: the mirror connected to the Reducing the voltage at the drain terminals of cells 15, 17 

lower-capacitance line (i.e., lower conductance — higher also provides for preventing or minimizing soft-writing of 

current) is turned on faster, so that, in the case in question, the cells, thus ensuring a high degree of reliability of the 

transistors 33 and 34 of section 31 are turned on faster than network. 

transistors 43 and 44, and begin drawing and supplying 35 By virtue of the low current levels and buffer 4 decoupling 

current more rapidly to node 37. More specifically, transistor the synapses and the neuron, the network provides for 

33 (and hence transistor 34) draw current I X =I C +AI. As only particularly high-speed evaluation. 

current 1 2 -I c is required, however, from node 37, the excess Forming the neuron as a feedback conductance 

portion supplied by transistor 34 tends to go to transistor 43, comparator, as described, provides for safely comparing 

which, not supporting such a condition, begins to go off, and, 40 even only slightly differing conductances in the synapses, 

at the same time, the voltage at node 37 rises to supply and in particular conductances differing by even a few orders 

voltage V 00 . of magnitude less than the conductances themselves (e.g., 

On the other hand, when turned on, transistor 44 supplies with a ratio of 1:1000), by virtue of the differential value of 

current to node 36, thus reducing the current through tran- the two conductances being used to off-balance neuron 5 and 

sistor 33 and causing a brief variation in the current supplied 45 hence latch circuit 6. 

by transistor 34 to node 37. On account of the cross- The network according to the invention also presents a 

connection of sections 31 and 32, a sort of feedback is high input range, can discriminate up to 10 bits, requires a 

produced between them, which causes the 30 section con- small integration area (200x32 fan) by using a 0.7 fan 

nected to the higher-current synapse block (in this case, CMOS technology, presents a simple, easily integratable 

section 31) to supply practically all the current required by 50 structure, and requires no interfacing with digital environ- 

synapse blocks 2, 3, while the other section is practically ments. 

turned off. In the case in question, therefore, at the end of the The present invention has further advantages as follows, 

transient state, practically all of current I C +AI flows into The disclosed conductance mode computation causes a 

section 31, and only a negligible current flows into section minimum current pulse in the summing node in order to 

32. By using a cross mirror and positive feedback on the 55 make neuron decisions. In addition, by virtue of minimum 

neuron, the present invention is able to subtract a common current pulse as well as of fast computing time, a low power 

mode current and discriminate lines of synapses with a operation is obtained. The solution by the present invention 

minimum conductance. This allows the circuit to achieve has high precision, in particular the disclosed solution has a 

high precision over a wide range of input current. The speed 9-bit inherent precision and a 7-8 bit measured precision, 

of the circuit changes with the overall current, therefore, to 60 FIG. 8 shows a test structure of an implementation of the 

optimize the speed performance and to digitize the output neuron. The test structure contains two pairs of transistors 

the present invention use a standard latch 6. connected to the neuron. One pair of the transistors are very 

The FIG. 7 plots show the currents in transistors 33, 34, big (B=800 fim/2 fan) and represent the common mode 

43, 44, as determined in a simulation test conducted by the signal for the positive and negative synapses; the other pair 

Applicant, upon switching of reset signals at the inputs 65 of transistors are small (S=0.8 /an/2 fan) like minimum flash 

of switches 29, 40, 41, and enabling of latch 6 (EN signal). devices and represents variable input signals. After offset 

As can be seen, at the end of the transient state, current I 33 compensation between the two large "common mode" 



